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Foreword 



The Achieving the Dream: Community Colleges Count 
Initiative represents a major force in a growing movement 
among higher education institutions to use data to 
improve student outcomes. This multiyear, national 
initiative, which is particularly concerned with student 
groups that have traditionally faced significant barriers to 
success, involves more than 80 colleges in 15 states. 
Achieving the Dream encourages participating colleges to 
engage faculty, student services staff, and administrators 
in examining data on student progress, in formulating 
strategies to address achievement gaps, and in 
evaluating those strategies to make them more effective. 
Participating colleges are also expected to use evidence 
of what works to promote student success as the basis 
for academic program review, strategic planning, and 
budgeting, thereby bringing to scale proven strategies and 
sustaining them over time. 

Launching such an extensive reform effort is an ambitious 
undertaking, especially for colleges little experienced in 
using student data beyond what is required for reporting 
to government and accreditation agencies. Colleges 
must not only develop the technical capacity to conduct 
the necessary research, they must also create a culture 
In which such research is valued and utilized. Many 
participating colleges are making progress, yet, because 
the initiative is still relatively young, few examples exist 
that actually illustrate how institutions have engaged in 
substantial organizational change based on analysis of 
what works to promote student success. 

Miami Dade College, the largest community college in the 
nation, is not participating in Achieving the Dream. Yet it 
is one of the small but growing number of institutions in 
the country that are taking a systematic approach to using 
data to inform college decision-making about student 
learning and performance. The account presented in 
these pages, told from the point of view of Miami Dade’s 
Office of Institutional Effectiveness, discusses how that 
approach took shape. Because of its size and the strength 
of its institutional research capacity, Miami Dade may have 



begun with some resource advantages that other colleges 
lack. At the same time, the college’s large size also meant 
that its leadership had to persuade stakeholders spread 
out among eight separate campuses to invest in the 
process. Finally, as the report makes clear, Miami Dade 
shares many of the same challenges as other community 
colleges, including large numbers of students unprepared 
for college-level work and a poor track record of student 
progression among students enrolled In remedial or 
developmental courses, especially in mathematics. 

Miami Dade began its reform by using a deliberative, 
stepwise approach to analyzing and acting upon 
problems with student achievement in one key discipline, 
mathematics. That approach has begun to show 
promising results, and it is now being applied to improve 
the impact of academic programs and student services 
throughout the institution. We hope that the story of Miami 
Dade’s experience in using data to promote student 
success is helpful to Achieving the Dream colleges 
and other Institutions throughout the country that are 
undertaking similar efforts to Improve student success. 

Thomas Bailey, Director 
Community College Research Center 
Teachers College, Columbia University 
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Assessing and Improving Student Outcomes: 
What We Are Learning at Miami Dade College 

Executive Summary 



This report describes how Miami Dade College (MDC) 
has begun to develop a systematic and data-informed 
method for assessing and improving student outcomes 
across its large eight-campus college. Told from the point 
of view of MDC’s Office of Institutional Effectiveness, the 
report recounts how the college has responded to the 
problem of poor student progression along the develop- 
mental/college-level mathematics pathway. 

Poor student achievement in mathematics is, of course, 
an issue of tremendous concern for many community 
colleges. At MDC, we have so far been encouraged by 
the effects of the strategies that have been undertaken 
over the past several years to address the problem. 

Some of the interventions that have been implemented 
are showing promising results. Equally important, the 
process that the college embarked upon to understand 
the roots of the problem, to act upon it, and to evalu- 
ate those actions has itself had a significant impact on 
faculty, administrators, and student support staff. It has 
to some extent been institutionalized in college-wide 
reform. Thus the approach taken to improve student 
outcomes in mathematics is now being used in other 
disciplines and program areas. It is also noteworthy that 
the strategies that the college researched and deployed 
to improve math outcomes were formalized in our re- 
accreditation-required Quality Enhancement Plan (QEP). 
The obligation to formulate the QEP and proceed by it 
have undoubtedly served to strengthen the commitment 
to improve student learning throughout the college. 

We have organized this report by describing the steps that 
were taken throughout the process. The steps build upon 
one another as themes, although events and practices are 
not always presented in strict chronological order. Some 
incidents that appear in different sections actually took 
place in the same period of time. 

Step 1 : Identify Problem Areas 

Different kinds of information coming from various sources 
confirmed for us that “college prep” and college-level 



math was a serious problem area throughout the college. 
(We refer to developmental courses as “college prep” 
courses at MDC.) Institutional analyses from the late 
1990s had shown, for example, that students needing 
remediation in math, regardless of other remedial needs, 
had lower rates of college prep completion than students 
who did not need remediation in math but who did need 
remediation in both the reading and writing subject 
areas combined. We also knew that MDC students were 
having a difficult time passing the mathematics subtest 
of the College Level Academic Skills Test (CLAST), an 
important state-wide exam that is commonly used to 
qualify students for the associate degree or for junior- 
status transfer in Florida. And we knew that, despite the 
guidance given by advisors, many students were delaying 
their enrollment in college-level math courses needed for 
graduation until the very end of their college tenure. The 
mathematics pathway was clearly not functioning as it 
was intended. 

Step 2: Gather More Data 

We began to look for more information that would help 
us figure out how best to help students. We began 
tracking students and found that students who take 
sequenced developmental math courses without any 
delays between courses do better than those who sit 
out one or more semesters along the way. Tracking data 
also suggested that, contrary to the speculation of some 
faculty members, curricular misalignment between the 
three courses in MDC’s developmental math sequence 
was a bigger problem than was flawed placement into 
developmental courses. 

Qur analytic findings were compelling for many faculty. 

Yet we also wanted to include more direct input from 
students themselves. We therefore held focus groups 
with developmental math students. The most striking 
comments heard in these focus groups involved 
embarrassment about asking questions in the classroom. 
Many focus group participants, in fact, felt that student 
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questions were largely unwelcome by their instructors in 
developmental math courses. This finding had a marked 
influence on members of the math discipline. 

Step 3: Build Forums for Collaboration 

By the fall of 2001 , a groundswell of interest among key 
math faculty had grown around the early information 
we had gathered about developmental math. As a 
consequence, the vice provost invited math faculty 
members at each campus to learn about and discuss 
the data we had collected. The faculty who attended 
these roundtable discussions were very engaged, and 
they found the data persuasive. Indeed they asked us to 
conduct further research (which we did) on questions they 
felt were important, such as whether there were student 
outcome differences based on full-time faculty versus 
adjunct faculty instruction (there were not). 

In 2003, the math faculty organized their first annual 
mathematics retreat to discuss relevant student data and 
possible interventions based on those data. In this first 
retreat, the math faculty focused on the issue of curricular 
alignment. They reviewed the content of the three 
developmental math courses. They decided to teach fewer 
competencies in each course and instead examine each 
associated topic area more deeply. They also redesigned 
the curriculum of each course so that the competencies 
taught in each were well aligned in the sequence. 

The roundtables and the first annual mathematics retreat 
spurred some math faculty to explore innovative ideas to 
promote student success. In addition, the idea of holding 
discipline-based retreats to discuss student data and 
student success has since spread to other disciplines and 
program areas. 

Step 4: Share Data Broadly 

We also began to share the data we had amassed about 
developmental math more broadly throughout the college. 
In order to reach multiple audiences, we share data we 



collect about student success issues through a variety of 
venues. For example, we give a data-informed presenta- 
tion about our student population and about particular 
challenges that students face at new faculty orientations. 
And we distribute easy-to-read institutional research sum- 
maries and recommendations for action, which are widely 
read by faculty and by the college and student leadership. 
The college also utilizes an online executive information 
system that provides queriable data on student perfor- 
mance and student feedback about instruction. (Nine 
terms of data are shown for most queries; thus, trends in 
pass rates, withdrawal rates, success rates, and student 
ratings are easy to identify.) Another way we share and 
collaborate is through what we call focused research, in 
which a group of faculty from a particular discipline work 
with institutional researchers in constructing research 
questions about student achievement. Finally, the college 
has initiated an internal grant process to encourage the 
development of specific strategies to address issues of 
concern about student success. Both faculty and student 
services staff may apply for modest grants. In 2006, 35 
different proposals were selected for funding. 

Step 5: Identify Strategies 

Primarily through the roundtables and retreats described 
in Step 3, several strategies were identified for improving 
student learning and student progression in mathematics. 
Faculty participating in the college’s re-accreditation went 
on to conduct more research on various interventions 
and then chose ten strategies for the college’s Quality 
Enhancement Plan (QEP), which was reviewed by the 
Southern Association of Colleges and Schools (SACS) in 
a 2004 site visit. The goal of MDC’s QEP is “to enhance 
student learning by developing innovative curricular, 
instructional, support, and assessment strategies in the 
high risk courses of College-Prep Mathematics, College- 
Prep Algebra, Intermediate Algebra, and College Algebra.” 
The QEP strategies are: 1) Establish a program of frequent 
testing; 2) Provide learning prescriptions for students who 
demonstrate need; 3) Establish mathematics advisement 
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procedures; 4) Provide supplemental instruction for 
students repeating college-prep math and algebra; 5) 
Implement a certified tutor training program; 6) Provide 
math success skills training for students repeating college- 
prep math and algebra; 7) Email personalized interim 
reports to students; 8) Establish a program to incorporate 
math into other disciplines; 9) Provide math faculty training 
on accommodating different learning styles; and 1 0) 
Upgrade math support labs. 

Step 6: Implement and Assess Strategies 

We insist that everyone who wants to implement a 
new improvement strategy at MDC have some kind of 
assessment plan to test how well that strategy is working. 
All the QEP strategies, for example, include assessments 
based on comparisons with historical data, comparisons 
of experimental and control groups, or comparisons of 
pre- and post-training ratings. Faculty committees headed 
by one lead faculty member are in charge of implementing 
each QEP strategy. 

Preliminary results on some of those strategies appear 
promising. The implementation of tutor training, for 
example, is associated with more frequent lab use and 
a modest increase in student satisfaction of math lab 
tutors. And since the improved math advisement was 
implemented, a higher proportion of students are taking 
their first required math course earlier than they did before. 
Likewise, more students are taking the next math course 
in their prescribed sequence immediately after completing 
the prerequisite course. 

We have had mixed results with the frequent testing 
strategy. We encountered some strange findings after 
the first round of implementation — the strategy seemed 
to be working for some sections of courses but not for 
others. We learned that faculty had not been following 
a consistent method in implementing the strategy. In 
action that we refer to as “closing the loop” — using 
implementation results to make changes in how a strategy 



is carried out — a decision was made to better define and 
manage how faculty apply frequent testing so that the 
strategy will be implemented with greater integrity. We also 
note that “closing the loop” in a series of implementation 
cycles corresponds to the notion of continuous 
improvement advocated by SACS and others. 

Step 7: Establish Accountability 

While MDC’s QEP and accreditation reports go a long 
way in assuring that the initial vigor of improvement efforts 
endure over time, in 2003 the college also adopted internal 
accountability requirements for the math discipline as well 
as for all MDC disciplines, schools, and student service 
areas. Each of these entities now prepares an annual 
report in which they articulate how they measure student 
outcomes. They also report on strategies they are using to 
improve student success and how, based on data, those 
strategies are working. The annual reports are an integral 
part of MDC’s strategic plan, which includes additional 
accountability measures associated with the college’s 
long-term goals. 

Step 8: Institutionalize the Process 

Annual reporting has become for the college a consistent 
and reliable method for disseminating information about 
the status of improvement strategies on a regular basis. 
The reports are reviewed by coordinating campus deans 
and presidents, and highlights are shared with the MDC 
executive committee so that decision makers are kept 
aware of the key strategies faculty are pursuing, what 
impacts the strategies are having, and how well these 
impacts match the strategic plan. Individual reflections 
about such efforts are also discussed in faculty and staff 
evaluations, thus emphasizing the importance the college 
places on promoting student success. Discussion and 
negotiation in a variety of venues — faculty meetings, 
executive committee meetings, new staff orientations, 
among others — is instrumental for institutionalizing the 
use of data to assess and improve student outcomes. 
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By incorporating the status of ongoing improvement 
strategies into standard review procedures such as annual 
reporting, the whole endeavor is better integrated into the 
regular functioning of the college. 

Conclusion: Lessons Learned 

We have learned a great deal in establishing this approach 
and broadening it for use in disciplines and program and 
service areas throughout the college. Throughout this 
report, we show how the use of data Is crucial not only for 
developing appropriate strategies to address problems 
such as poor student progression, but also for determining 
if such strategies are effective, and for modifying those 
strategies when necessary. Several of the most Important 
lessons we have learned so far relate to how data are used 
and shared. 

1) Start with available data and dig deeper as necessary. 
We had long known that developmental math was 
not working well. We gathered more data so we could 
make valid assessments and choose appropriate 
interventions. 

2) Use a variety of measures and methods. Cross- 
sectional analyses, longitudinal analyses, and 
benchmark comparisons can all provide insights. 

We found that Including student voices through 
focus groups helped us both understand the 
challenges and garner support for change. 

3) Share, listen, and collaborate with stakeholders. 

This is especially important for institutional 
researchers. Our researchers participate In various 
meetings throughout the college and try to Interact 
with faculty and staff as interested helpers rather than 
as auditors. 

4) Establish assessment plans before implementing 
strategies. In order to measure the effects of a given 
strategy, a method of assessment that addresses 



how that strategy contributes to improved student 
outcomes must be established beforehand. We 
found such assessment results crucial In recognizing 
problems in the implementation of the college’s 
frequent testing strategy. 

5) Channel the enthusiasm for new ideas. Once core 
strategies have been established, it is important to 
ensure that they are implemented and assessed with 
rigor throughout a complete cycle. Turning too much 
attention to other ideas that might well hold merit can 
serve to jeopardize existing efforts. 

6) Integrate Innovation with existing college processes. 
Whenever possible, MDC personnel used standard 
review procedures as channels for proposed 
innovations that required various approvals. Working 
within the traditional college structure made the 
whole endeavor easier to carry out. 
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Introduction 

This report tells how Miami Dade College (MDC) is using 
the interpretation of a variety of data in order to make bet- 
ter-informed choices about the operation of student pro- 
grams and services. MDC has made a number of changes 
in an attempt to improve student outcomes across this 
large, eight-campus college. While the endeavor continues 
to move forward, some specific strategies relevant to the 
college’s developmental mathematics pathway that were 
launched a few years ago have begun to show promis- 
ing results. What is more, the processes that took place 
in choosing, implementing, and assessing those strate- 
gies have to some meaningful extent been institutionalized 
through college-wide reform. Thus the methods of inquiry 
and action that have been applied to math are now being 
used in other disciplines and program areas. 

In this report, we elaborate upon a number of “steps” 
that were taken to improve developmental and college- 
level math outcomes at MDC, beginning with the 
identification of math content as a serious area of 
concern for a large portion of the college’s student 
population. We use the simple term “steps” with some 
caution, however. While the steps that we name in 
organizing this report are intended to express a causal 
chain of actions in terms of progress toward the goal of 
improved student outcomes, the events we describe are 
arranged thematically rather than in strict chronological 
order. Moreover, the steps we discuss should also 
be considered iterative. They can be, and have been, 
revisited multiple times in whatever order circumstances 
demand, forming feedback loops that accommodate new 
insights or that re-establish relevant facts or collective 
goals for faculty and staff. The steps are also iterative in 
the sense that they can be repeated in cycles to make 
cogent modifications to interventions that have already 
been adopted or to respond to new challenges. In this 
way, the steps function like action stages in a model for 
organizational change. 

The eight steps we sketch out are these: 

Step 1 . Identify problem areas 

Step 2. Gather more data 

Step 3. Create forums for collaboration 

Step 4. Share data broadly 

Step 5. Identify strategies 

Step 6. Implement and assess strategies 



Step 7. Establish accountability 
Step 8. Institutionalize the process 

Before discussing these eight steps, we set the stage by 
introducing Miami Dade College and the characteristics 
of its students. In a final section, we conclude this report 
by reviewing some important lessons we learned as a 
result of this undertaking. 

We tell this story from the perspective of the Office of 
Institutional Effectiveness (IE) at MDC. The IE office, 
made up of twelve staff members, is responsible for 
institutional research (IR), outcomes assessment, 
placement and exit test administration, student feedback 
surveying, and strategic planning. The IE office also 
provides extensive support for accreditation review. 

Miami Dade College and its student 
demographics 

Miami Dade College first opened its doors in 1960 
on what is today known as the North Campus, near 
the Opa-Locka Airport. Originally named Dade Junior 
College, MDC has grown into the largest institution of 
higher education in the United States, serving well over 
150,000 students per year and employing 8,300 persons 
in total, including 2,400 faculty members. This state- 
supported college has eight campuses and offers more 
that 150 associate degree programs. It maintains more 
than 50 transfer agreements with colleges in and outside 
Florida. While the college has, since 2003, begun to 
award bachelor’s degrees in a handful of programs, it still 
functions primarily as a community college. More than 
95 percent of its credit-earning students are Miami-Dade 
County residents. They pay Florida resident tuition, which 
is $69 per credit for lower division classes. 

College mission and strategic plan. MDC’s mission 
statement focuses explicitly on its students and the 
surrounding community: 

The mission of Miami Dade College is to provide 
accessible, affordable, high quality education 
by keeping the learner’s needs at the center of 
decision-making and working in partnership with its 
dynamic, multi-cultural community. 

MDC’s strategic plan focuses on operational aspects 
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of the college mission. It identifies critical issues and 
sets long-range goals along five themes: access to the 
college, student achievement and success, serving the 
community, resource development and utilization, and 
employees and the college. 

Enrollment. MDC currently enrolls about 158,000 students 
annually (about 80,000 are credit, degree-seeking 
students, and about 78,000 are in non-credit programs). 

Race/ethnicity and citizenship. About 66% of credit 
students are Hispanic; 21 % are Black Non-Hispanic. 
About 62% of credit students are U.S. citizens; about 
29% have Resident Alien status; and almost 6% are 
refugees or asylees. MDC’s overall student body comes 
from 182 different countries and speaks 93 languages. 
The greatest number of non-U. S. citizen credit students 
hold citizenship in Cuba (6,800), Haiti (2,600), Colombia 
(2,500), and other Latin American countries. 

Gender. About 61 % of credit students are female. 

Age. The average age of credit students is 26 years. Only 
31 % are the “traditional” college age of 18-20 years. 
About 36% are age 26 or older. About 62% of vocational 
certificate students are age 26 or older. 

Socioeconomic status and employment. About 51 % 
of credit students are first-generation college students. 
Only 30% of credit students have parents who earned 
bachelor’s degrees. The majority of credit students are 
employed, with 30% working 40 hours per week or more. 
About 35% of credit students are below the poverty 
threshold for their household size; about 57% are low 
income (150% of the poverty threshold). About half 
the students obtain financial aid in the form of grants, 
scholarships, loans, or employment. 

College goals and part-time status. Most MDC credit 
students come to college to earn a degree. About 65% of 
credit students seek an A.A. (transfer) degree; about 24% 
seek a career oriented 2-year A.S. degree; about 8% 
seek a credit certificate in short term career programs 
or are taking courses to upgrade skills or prepare for 
transfer. About 66% of credit students are enrolled part- 
time. About 26% of students take most of their classes 
outside the “traditional” weekday school hours. 



Issues that typically face MDC learners 

While MDC is by any measure a very large Institution, its 
learners are similar to those in other community colleges. 
Many MDC students face barriers to achievement 
associated with academic underpreparation, poor 
English skills, delayed entry to college, outside priorities 
such as a job or the care of dependents, and, finally, 
unfamiliarity with college and the benefits that college 
credentials provide in meeting career goals. Many 
students, in fact, face a multiplicity of such barriers. 

Inadequate preparation. As in most community 
colleges, many of MDC’s entering students are 
academically underprepared. Only about 20% of 
students enter MDC ready for college-level work, as 
indicated by our Computerized Placement Test (CPT). 
The great majority (80%) of incoming students need 
what MDC calls college preparatory courses (outside 
MDC, they are more commonly known as developmental 
or remedial courses) or ESL courses before beginning 
college-level coursework. About 25% of incoming 
students need remediation in all three basic skills areas 
— reading, writing, and math. At MDC, our mathematics 
“college prep” sequence is housed within and taught by 
the college-wide mathematics discipline. Our reading 
and writing college prep sequence, in contrast, is taught 
by a stand-alone college-wide reading and writing 
college prep discipline. 

Weak English skills. About half of MDC students are 
non-native English speakers. About 20% of students 
begin in ESL before they take the CPT and move on 
to developmental or college-level courses. Whether or 
not they need ESL, many non-native English speaking 
students are challenged to complete college-level 
coursework in English. They may speak well, but they 
may not read or write as well as they should for engaging 
in course content. Again, this is a problem shared by 
students at many other community colleges. 

Delayed entrance. Most incoming MDC students are 
delayed-entry students; that is, they have stayed out of 
school for more than one year before arriving on an MDC 
campus. Such students have lost momentum in learning, 
and they may have forgotten more of what they learned 
in high school than traditional-age students. 
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Employment and family responsibilities. Just over half 
of students are from low-income households. Needless 
to say, this economic insecurity creates worries and 
disruptions and reduces the time students have to 
devote to their studies. For many of these students, 
their first priority is earning enough money to contribute 
satisfactorily in paying household bills. College is generally 
not the first priority for them. As they navigate the low- 
wage labor market, they “fit college in” where they can. 
About 80% of students work part time; about 30% work 
full time. In addition, many of these same students have 
family responsibilities, such as taking care of children or 
parents. So it is not unusual for students to have a job 
as well as significant family responsibilities while they are 
taking classes. 

Unfamiliarity with college life. Many low-income students 
are also first-generation college students. A large number 
of these students are wholly unfamiliar with college life. 
They often have poor study habits and lack clear goals 
for postsecondary education and employment. MDC 
offers a first-year Student Life Skills (SLS) program aimed 
at students who need help in addressing these issues. 

The program includes “student success” courses that 
teach study skills, time management skills (including the 
necessity of classroom commitment and of scheduling 
study time for each class), goal setting, and planning. 

A great deal of effort in the SLS program is aimed at 
having students recognize the value of a college education 
in terms of the increasing skills and credentials that 
are needed for today’s careers. This exercise is more 
complicated than simply informing a given student about 
the advantage of having a degree in the labor market. For 
a low-income, first-generation student may also receive 
conflicting messages from respected family members at 
home. The student might hear that college is something 
“extra” beyond what is really needed to attain a good job 
or that college is “not for everyone.” The student might 
also encounter complaints about not contributing enough 
to pay for shared household expenses. Thus, dispelling 
the myth that college is frivolous or that college will not 
“get you anywhere” is not an easy task, especially for 
students who do not have close ties to anyone who has 
graduated from college. 

First-generation students often need help in making 
connections to other students and to faculty and staff on 



campus so that they experience a sense of belonging. 
Each student in the SLS program develops a strong 
connection with an SLS faculty member, who serves 
as advisor and mentor in many cases. This relationship 
is often fundamental in having students slowly gain 
confidence and take on the belief they belong at MDC. 

The Student Outcomes 
Improvement Process 

Step 1 : Identify problem areas 

At MDC we have many different ways to look at what 
the college does well and what could be improved upon 
in terms of educating students. For example, we have 
access to longitudinal tracking data with Florida college 
comparisons, results from the Florida Basic Skills Exit 
Test and the College Level Academic Skills Test (CLAST), 
general education outcomes assessments, enrolled 
student and graduate surveys, transfer student success 
reports, and anecdotal reports, to name a few. 

Core indicators report. One of the most important ways 
we look for problem areas in student achievement is 
through our MDC core indicators report, a kind of global 
report card that was developed in accordance with our 
college mission in 2003-2004. The core indicators report 
alerts the college executive committee and the academic 
and student leadership to situations in which MDC is 
not doing as well as it should, based especially on data 
comparisons with other institutions in the state. (MDC 
uses state accountability measures and participates in the 
national community college benchmarking project.) 

Several different categories of indicators are shown on 
the core indicators report, including those that address (1) 
student access (what kinds of students MDC draws from 
high schools, and whether class locations and schedules 
are convenient), (2) student affordability (including the 
average percentage of student family income spent on 
education), (3) student success and progression (which 
features many indicators, including data on the proportion 
of students who complete “college prep” in two years, 
and how well those same students do in their first 
college-level courses, as well as retention and graduation 
rates), (4) success after leaving the college (e.g., transfer 
student performance and job placement data), (5) student 
satisfaction, (6) employees and the college (taken from 
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survey ratings, turnover rates, and training investments), 

(7) how weil the coilege serves the community (minority 
participation rates, market penetration, service learning 
contributions), and (6) the efficient use of resources 
(including class size, student/faculty ratio, space 
utilization, grant funding). 

Low college prep graduation rate. It was by looking at 
student success and progression indicators in about 2001 
that it became clear to us that, first, MDC’s graduation 
rate was not quite as high as the state average and that, 
second, MDC’s graduation rate for those students needing 
remediation was not as high as the state average for 
such students. While our “college ready” rate compared 
favorably with the state average, our “college prep” rate 
was slightly lower than the state average. According to 
subsequent statewide analysis by the Florida Department 
of Education on the cohort of fall 2001 first-time-ln-college 
students, MDC’s overall five-year graduation rate was 
23.2 percent, compared to 24.0 percent for the statewide 
community college system. The MDC graduation rate for 
college-ready students was 42.5 percent, compared to 
40.9 percent statewide. For students who needed some 
kind of remediation, the MDC rate was 18.5 percent, 
compared to 1 8.8 percent statewide. For students who 
needed remediation in all three areas, the MDC rate was 
9.8 percent, compared to 1 1 .7 percent statewide. 

Low college prep math program completion rate and 
low college prep math graduation rate. Armed with this 
fact, we delved deeper to figure out why it was that MDC 
students needing remediation faired slightly worse in terms 
of college completion than students needing remediation 
at other institutions. We knew that a significant hurdle for 
MDC “college prep” students was completing college prep 
In the first place. And the data showed that the college 
preparatory skills area with the lowest rate of completion 
was math. This was not a surprise to us. In the late 1990s, 
institutional analyses at MDC had indicated that college 
preparatory math was an area of concern. Those analyses 
had shown that fewer than 30 percent of students scoring 
at the lowest level on the mathematics placement subtest 
completed the college preparation program within three 
years, no matter what their score on the verbal placement 
subtests. In contrast, 40 percent of students scoring at the 
lowest level on both verbal subtests (reading and writing) 
completed the college preparatory program within three 
years, as long as they were not also deficient in math. 



(Students scoring at the lowest level on only one of the 
two verbal subtests had a 52 percent three-year college 
prep completion rate.) 

Analysis by the Florida Department of Education of the 
graduation rates for various student groups in the fall 
2001 cohort also showed differences among college 
prep program areas: the five-year graduation rate among 
MDC students needing remediation in math only was 
23.3 percent, compared to 37.6 percent for reading only 
and 37.1 percent for writing only. For students needing 
remediation in math and one other subject area, the MDC 
graduation rates were 19.3 percent (math and reading) 
and 1 5.5 percent (math and writing), compared with 32.4 
percent for students needing remediation in both reading 
and writing. Clearly, college preparatory math was a 
problem. 

We continued to compare MDC’s data to state system 
data to see If other factors were relevant. We came to 
realize that part of the reason that we lagged behind 
the state was that a higher proportion of our students 
needed remediation in all three skills areas, so MDC 
students needed more remediation, at least by these 
measures, than the state average. It is also the case that 
MDC offers multiple levels of remediation (in math, MDC 
offers three courses: college prep arithmetic, college prep 
mathematics, and college prep algebra). We speculated 
that MDC probably starts students at a lower level of 
remediation than do some other colleges in the state. 

This would mean that it takes longer for some of MDC’s 
students to complete college prep and move on to 
college-level courses compared to students at colleges 
with fewer levels of remediation. Experiencing a longer 
delay until the start of college-level courses could affect 
graduation rates as well. 

While student progression and success indicators, which 
now appear on the core indicators report, were the main 
sources of data in driving early discussion that identified 
developmental math as an Important problem area that 
we wanted to focus on, we also had access to other data 
which were meaningful, some of which were available as 
much as ten years ago. 

Poor sequencing In math course enrollments. For 
example, our longstanding knowledge of MDC students’ 
frustrating experiences with the CLAST (particularly the 
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mathematics subtest) provided important ciues about 
difficulties in math, and it guided more anaiysis about 
when students tend to take their required college-levei 
math courses during their coiiege tenure. The Coilege 
Level Academic Skills Test (CLAST) once carried more 
weight than it does today (there are now curricular 
alternatives to taking or passing the CLAST). Until the 
late 1 990s, all Florida community college students were 
required to pass all four parts of the test in order to qualify 
for the A.A. degree or to transfer as a junior to a four-year 
college. Attaining the minimum score needed on the math 
portion of the CLAST was the most significant problem 
MDC students had with the test. In far too many cases, 
students were ready in every other way to graduate, but 
they still had yet to pass the math portion of the CLAST. 

The MDC IR office conducted some transcript analyses 
and cross-sectional analyses to gain more information 
about this issue. In so doing, we discovered that many 
students were postponing taking required math courses 
until the end of their tenure at MDC. Even more troubling, 
we learned that in some cases in which students needed 
only two or three more courses to fulfill their graduation 
course requirements, those two or three courses were, 
in fact, required math courses that had to be taken 
sequentially. This meant that students were sometimes 
required to spend an extra semester or even a year to 
complete their math courses 
after having completed all 
other coursework. This was no 
surprise to many of our faculty 
and advisors who struggled 
to persuade students to take 
math courses earlier, as these 
courses were often the ones 
that caused the most anxiety for 
students. Clearly we needed to 
find effective ways to compel 
students to take college-level 
math courses sooner in their 
community college tenure. 

Step 2: Gather more data 

Having recognized that math 
was a problem area, and 
having identified some specific 
problems that required action. 



we began to look for more information that would help us 
figure out how best to help students. State analyses (such 
as the one discussed above) turned out to be helpful, as 
were student focus groups that we conducted on several 
MDC campuses. At the same time, much of what we 
continued to look at was more student progression and 
success data. 

It is worth noting that while the collection of these 
data took place before we began to meet in broad 
collaboration with faculty in formal meetings, this complex 
set of undertakings did not occur in isolation. It took 
place in consultation with academic deans and faculty 
members and, in particular, with key faculty from the 
math discipline. The fact that the gathering of data was a 
shared effort on the part of both institutional researchers 
and representatives of this academic discipline is 
very important. Not only was the cross-fertilization of 
knowledge and ideas useful, but the fact that we engaged 
in a collective and open effort meant that there was never 
a time when results were “unveiled” for an unwitting 
faculty. Rather, faculty members were integrally involved 
in formulating research questions and in learning from the 
findings as the data emerged. 

High risk courses. For nearly ten years we have been 
identifying what are sometimes called “high risk” or 



Table 1 

High Risk Courses, Fall Term 2003 


Course 


Enrollment 


Pass Rate 


BSC 


1005 


Principles of Biology 1 


1,883 


59.7 % 


LIT 


2120 


Survey of World Literature 


663 


59.3 % 


CHM 


1025 


Introduction to Chemistry 


692 


59.2 % 


BSC 


2085 


Human Anatomy and Physiology 


2,147 


59.1 % 


ACG 


2021 L 


Financial Accounting Lab 


1,049 


58.9 % 


MAC 


2311 


Calculus and Analytical Geometry 1 


442 


57.5 % 


MAC 


1105 


College Algebra 


4,665 


53.8 % 


MAC 


1140 


Pre-Calculus Algebra 


516 


51 .9 % 


GLY 


1001 


General Education Earth Science 


603 


51.1 % 


ANT 


2410 


Introduction to Cultural Anthropology 


322 


50.6 % 


MAT 


1033 


Intermediate Algebra 


5,979 


50.1 % 


MAT 


0002 


College Preparatory Arithmetic 


1,706 


49.9 % 


MAT 


0024 


College Preparatory Algebra 


2,414 


48.7 % 


MAT 


0020 


College Preparatory Mathematics 


4,106 


47.1 % 


MAC 


1114 


Trigonometry 


409 


42.8 % 



Notes: The pass rate is equal to the number of A, B, C, and S (satisfactory) grades divided by the total number 
of all grades, including withdrawals. 
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Table 2 








Comparison of Pass Rates of Students Who Enrolled in a Course 
Immediately After Passing the Prerequisite in the Previous Term 


and Those Who Delayed (Spring Term 2002) 




Course 


Immediately 


Delayed 
One Term 


Delayed 
Two Terms 


MAT 0024 (from MAT 0002) 


38% 


29 % 


20 % 


MAT 1033 (from MAT 0020) 


46% 


32 % 


29 % 


MAT 1033 (from MAT 0024) 


53% 


35 % 


33 % 


MAC 1105 (from MAT 1033) 


57 % 


50% 


50 % 



“gatekeeper” courses at MDC. We 
define these as large-enrollment 
courses that have a pass rate 
that is significantly lower than the 
average pass rate at the college. 

We identify high risk courses each 
year and show the results to the 
college leadership and faculty. In 
2003 we identified 15 high risk 
courses (see Table 1). Eight of 
these courses were in the math 
discipline, and they included all 3 
of the college prep math courses. 

More recently, we have begun to ask more pointed 
questions about high risk math courses: Are students 
being placed into them without adequate preparation 
to be successful? Are students coming out of a 
prerequisite course without having gained competencies 
to succeed in the next “high risk” course in the sequence? 

We began tracking students to see what level of math 
remediation they start in, how they progress, and what 
percentage of them in each cohort completes the math 
requirements they need for their degree (generally two 
college-level math courses) within three years. Using 
such tracking data, we are able to see where and to some 
extent why we lose students along the mathematics 
pathway. Are we losing students after the first remedial 
course they take because they did not pass that course? 
Or are we losing them after the first course because they 
did not enroll in the second course in the sequence? Or 
are we losing them because they took but did not pass 
the second course? 

Immediate versus delayed progression. Much of our 
progression and success analysis focuses on course 
sequencing and the effects of taking breaks in such 
sequences. We can compare the pass rates for immediate 
versus delayed progression. For example, we looked at 
how much of a difference it makes if, after taking the first 
developmental math course, students sit out one or two 
semesters or longer before enrolling in the next course 
in the sequence. We found that it makes a considerable 
difference (see Table 2). If students take sequenced 
developmental math courses without any delay, they do 
better. While this result is not a big surprise — skills and 
concepts learned for the first time in one course might be 



forgotten if the next course that utilizes them is delayed for 
very long — it is an important finding that we have verified 
and passed along to our faculty and advisors so they can 
better persuade students enrolled in developmental math 
to complete the sequence in consecutive semesters. 

CPT efficacy and curricular alignment. We can also 
make another kind of comparison using progression 
analysis. We can learn what percentage of those 
students who made an immediate (rather than delayed) 
succession to their second developmental math course 
actually pass that second course compared to those 
students who were placed, by virtue of the CPT, directly 
into that second course. 

This comparison is important at MDC because some of 
the faculty believed that the passing scores on the CPT 
were not set correctly, and that students were being 
placed into math courses for which they were not truly 
prepared. But, in fact, when we looked at the results of 
progression analyses, we saw that the students who 
tested into the second level developmental math course 
actually did better than those who came up through the 
prior developmental course (see Table 3, next page). This 
means that a different issue is at play than what some 
faculty suspected. The greater problem involves not the 
CPT but rather how students progress between levels of 
developmental math. The data suggest that there may 
be some curricular misalignment in the developmental 
sequence, because many students who come out of the 
first developmental math course appear inadequately 
prepared for the next course. 

State system research on SLS courses. State analyses 
have also provided us with useful information. In particular. 
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Table 3 

Comparison of Pass Rates of Students Plaeed by the CPT and 
Those Who Passed the Prerequisite (Spring Term 2002) 


Course 


Placed 
by CPT 
Scores 


Passed 
Prerequisite 
Prior Term 


Gap 

(Difference) 


College-Prep Algebra (MAT 0024) 


65 % 


38 % (from MAT 0002) 


27 % 


Intermediate Algebra (MAT 1 033) 


60% 


46 % (from MAT 0020) 


14% 


Intermediate Algebra (MAT 1033) 


60% 


53 % (from MAT 0024) 


7 % 


College Algebra (MAC 1 1 05) 


76% 


57 % (from MAT 1033) 


19% 



the Florida state system’s study of Student Success 
courses (which are called Student Life Skills, or SLS, 
courses in Florida) has helped us to better target subject 
area deficiencies. Using longitudinal student data, the 
state’s analysis (Florida Department of Education, 2006; 
see also Zeidenberg, Jenkins, & Calcagno, 2007) found 
that SLS courses do, in fact, boost outcomes of students 
who do and who do not place into college prep. The 
state’s researchers also broke their results down by 
subject area to see if SLS courses were just as helpful for 
students needing remediation in math as they were for 
students needing remediation in reading or writing. They 
found that SLS was not as helpful for developmental 
math students as it was for others. 

This fact supported a decision the college had made in 
2002 to develop a linked course option in which students 
taking a developmental math course also enroll in an 
SLS course during the same semester. The curriculum 
of these two courses is connected in an attempt to 
make the SLS content more relevant to college prep 
math students. For example, when the theme in the SLS 
course is effective test-taking strategies, instructors 
use examples from the math course to teach those 
strategies. (Early indications are that these two courses 
really do support one another, leading to improved 
progression and success in developmental math. For 
example, pass rates for several linked mathematics 
courses that were evaluated in 2004 were 10 or more 
percentage points higher than the same course taught on 
the same campus with approximately the same number 
of students without the link to the SLS course. Similarly, 
re-enrollment rates for the following term were 5 or 
more percentage points higher for students in the linked 
mathematics classes.) 



Student focus groups. Another 
Important way we gained 
understanding was by conducting 
focus groups with developmental 
math students in 2001 . (Given 
the value of what we learn from 
focus group discussions, we have 
conducted numerous focus groups 
with other kinds of students since 
that time.) As described in this 
report, a picture based largely 
on numerical data was already 
emerging before we held these 
first focus groups: Students in need of math remediation 
do not do well In either developmental math or in college- 
level math courses; they tend to put off enrolling in 
math courses because they find them frustrating; they 
take breaks in between required math courses; and the 
developmental courses themselves are probably not 
aligned as well as they could be. Yet we also wanted to 
hear what students had to say about developmental math 
instruction and available support services. 

We convened groups of developmental math students 
and asked questions such as these: What is helping you 
in these math courses? What are the instructors doing 
that is helpful? What are they not doing that would be 
helpful? Do you feel you are adequately prepared for 
each math course? Did the CRT put you in the right 
place? What do you think is important for success in 
these courses? 

We heard a great deal from students. The most striking 
thing students told us — something that we now 
emphasize as a key point in new faculty orientation 
meetings — is how extraordinarily important it is for them 
to feel comfortable when asking questions in class. Many 
students reported feeling embarrassed when asking 
questions. They also reported feeling that their questions 
were not welcomed by instructors. Indeed, some students 
perceived that instructors gave the impression that if 
students do not already understand some basic concept 
relevant to a lesson, they should not even be In the class, 
let alone asking questions. Whether or not instructors 
actually meant to impart this impression, the students’ 
perceptions of the dynamic is nonetheless meaningful and 
important. It is easy to see how this perceived dynamic 
could thwart student engagement in classroom learning. 



12 



Assessing and Improving Student Outcomes at Miami Dade Coilege 



Students in those first focus groups aiso told us 
that attendance is very important for success in 
developmentai math courses because it is remarkabiy 
easy to “fail behind.” Some noted that missing only one 
or two classes can mean having to struggle Intensely to 
catch up. Unfortunately, missing an occasional class is 
not uncommon for students who have significant work 
obligations outside college; their scheduled work hours, 
we learned, are sometimes extended by employers with 
little or no warning to meet unexpected demands. 

The students also told us that it would be helpful if 
faculty allowed students to work through some of their 
“homework” problems in class so that they could receive 
more guidance or get specific questions answered 
that might otherwise arise later in attempting to work 
through problems alone. Finally, the students noted that 
they found the math support labs to be very helpful, but 
thought the lab tutors could be more knowledgeable. 

Step 3: Create forums for collaboration 

Discipline-based roundtables and day-long discipline 
retreats with math faculty became the original major 
venues for sharing the data we were collecting and 
for collaborating with math faculty as a discipline. It is 
important to recognize that there was a broad overlap in 
terms of the gathering of data described in Step 2 and 
the sharing of data we describe below. Many activities 
and events discussed in Steps 2 and 3, and even Step 4, 
actually occurred during the same period of time. 

Roundtables. By the fall of 2001 , a groundswell of interest 
among key math faculty had grown around the early 
information we had gathered about developmental math. 
The vice provost for education therefore called together 
the math faculty in the first of a series of roundtable 
discussions. The vice provost invited faculty members at 
each campus, some of whom had not been involved with 
us at all, to learn about and discuss the data we had thus 
far collected. By this time, it was not difficult to persuade 
the faculty about the importance of our work. Throughout 
the college it was widely believed that achievement in the 
developmental math and early college-level math pathway 
was the single greatest challenge that our students faced 
on the road to graduation. Indeed the math faculty were 
probably growing weary of hearing that “problems in 
math” were a principal reason why MDC students were 



not succeeding in greater numbers. The math faculty who 
attended the roundtables were thus very willing to hear 
about our findings and to discuss what steps should be 
taken in light of this evidence. 

We were also pleased that the data we shared in these 
first formal discussions were found to be persuasive 
among the attendees. Not only did the faculty accept the 
validity of the findings, they asked us to conduct further 
research on questions that they felt were important, which 
we did. For example, they asked us to find out whether 
students taking courses taught by full-time faculty had 
better outcomes than students taking courses taught 
by adjuncts. (This did not turn out to be true.) They also 
wanted to see more analysis related to placement testing 
procedures. For example, they were concerned that taking 
a practice test and “brushing up” prior to taking the CRT 
resulted in course placements that were too high for 
students’ skill levels. (This also did not turn out to be true.) 
Finally, faculty wanted to review the success rates for 
students who were repeating math courses for the second 
or third time to see if they continued to struggle. (Analyses 
showed they did.) 

The findings that spawned the most conversation among 
faculty at the roundtables were those that came from 
the developmental math student focus groups, and in 
particular the finding about how students perceived 
instructor attitudes about in-class questions. Many faculty 
were astonished that students “don’t think we want them 
to ask questions,” and some were initially incredulous. Yet 
it is likely that upon reflection this finding strongly affected 
the faculty. Some faculty members reported that they 
could imagine how some instructors — and perhaps even 
they themselves at times — might come across in the 
way that these students perceived. Including the student 
voices in the data we shared undoubtedly made a strong 
impression on the faculty. 

Annual retreats. In 2003, the math faculty organized 
the first mathematics retreat in order to take a closer 
look at curricular alignment in the developmental math 
sequence and to confer about related matters. Their 
discussions resulted in action to improve the content 
of the developmental courses. They decided to teach 
fewer competencies in each course and instead explore 
each of the associated topic areas more deeply. They 
also redesigned the curriculum of each course so that 
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the competencies taught in each were weli aligned in 
the sequence. 

In the months that followed both those first roundtables 
and the first retreat, a number of math faculty who had 
attended became much more enthusiastic about making 
changes that would help students along the mathematics 
pathway. In subsequent years, these faculty have 
organized and held annual discipline-wide retreats to 
discuss problems, strategies, and the results of actions 
taken. They have helped to push strategies forward and 
to elicit greater interest on the part of their colleagues 
and especially new faculty. Institutional researchers 
provide data to support discussions and attend these 
annual retreats. We share whatever new institutional 
data we have, including data that speak to how well their 
strategies are working. 

Since the launch of the annual math retreats, math faculty 
have begun to explore innovative ideas to promote 
student success. For example, after the first retreat, a few 
faculty who teach algebra decided that students might 
do better if they interacted with each other more deeply 
in problem-solving. These faculty replaced the desks in 
their classrooms with round tables, and they introduced a 
great deal of project-based instruction during classtime. 
Students were obliged to face each other and discuss 
how to work through problems, resulting in a more 
collaborative atmosphere in the classroom. 

The idea of holding retreats to discuss student data and 
possible interventions based on those data has begun to 
spread to other disciplines and program areas. The ESL 
discipline and the nursing area have each held a retreat, 
and the college prep discipline (for reading and writing) is 
planning one in the near future. In addition, all disciplines 
hold a discipline meeting at least once per term. And each 
year they spend part of a professional development day 
in discipline discussions, which typically focus on student 
data and student improvement issues. 

Step 4: Share data broadly 

By 2003, we had learned a great deal about issues 
relevant to the developmental math pathway. Having 
gathered so much relevant data, we began to share it 
more widely with faculty college-wide. The IR office shares 
data in different ways, ranging from hard-copy and web- 



based distribution of research findings to PowerPoint 
presentations and face-to-face discussions with faculty In 
a variety of venues. 

New faculty orientations. One such venue is new faculty 
orientations. In these orientations, we use data to provide 
an overall picture of our student population as well as 
to point to particular areas of concern. We use data to 
tell incoming faculty about what we see In terms of the 
students’ greatest challenges and what the faculty’s role 
might be in helping them meet those challenges. In the 
past few years, a portion of each orientation has been 
dedicated exclusively to developmental math. This is due 
both to the concern about developmental math already 
described and to the related fact that the faculty chose 
four high risk courses from the developmental math 
and early college-level math pathway as the topic for 
MDC’s Quality Enhancement Plan (QEP), a college-wide 
faculty-led plan for Improving student learning that the 
college developed in its last round of accreditation by the 
Southern Association of Colleges and Schools (SACS) (the 
site visit was in 2004). More about the college’s QEP and 
SACS accreditation is described below. 

Hard-copy and computer-based IR reporting. We 
also disseminate IR findings to the appropriate audience 
of college employees and others in the form of reports 
and briefing packages. Figure 1 shows an example of 
how student progression data are displayed in an MDC 
briefing package. These packages are prepared annually 
for each campus and the college with information that the 
presidents and leadership teams are most likely to need. 

IR findings are also disseminated through “capsules” 
and “spotlights” — very short, easy-to-read summaries 
of research findings and recommendations for action, 
which are widely read by faculty and staff as well as by the 
college and student leadership. In addition, we maintain 
an easy-to-navigate IR website, where faculty can go to 
get more detailed Information about IR studies, including a 
page devoted to “focused research” specific to disciplines 
and programs (discussed below). 

The college also utilizes an online executive information 
system, which includes a feature that provides queriable 
current and historical data on student performance and 
on student feedback about instruction. This system may 
be more well-developed at MDC than similar systems at 
other community colleges. It presents different levels of 
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Figure 1 

Sample of Data on Student Progression Displayed in an MDC Briefing Paokage 



How well are MDC students progressing through Math courses? 



MAT 0002 Enrollment 
and Pass Rates 




Successful Students' 
Progression to MAT 0024 


Success in MAT 0024 




Fall 

2003-1 


Fall 

2004-1 


Fall 

2005-1 


Fall 

2006-1 


Spring 

2003-2 


Spring 

2004-2 


Spring 

2005-2 


Spring 

2006-2 


Spring 

2003-2 


Spring 

2004-2 


Spring 

2005-2 


Spring 

2006-2 


1,706 


1,883 


1,834 


1,736 


631 


577 


608 


603 


220 


222 


193 


240 


49.9% 


41 .3% 


44.2% 


44.8% 


74.1% 


74.2% 


75.1% 


77.5% 


34.9% 


38.5% 


31 .7% 


39.8% 












Direct Placement Success: 


62.9% 


61.1% 


56.2% 


56.2% 



MAT 0020 Enrollment 
and Pass Rates 




Successful Students' 
Progression to MAT 1 033 


Success in MAT 1 033 




Fall 

2003-1 


Fall 

2004-1 


Fall 

2005-1 


Fall 

2006-1 


Spring 

2003-2 


Spring 

2004-2 


Spring 

2005-2 


Spring 

2006-2 


Spring 

2003-2 


Spring 

2004-2 


Spring 

2005-2 


Spring 

2006-2 


4,106 


4,262 


3,922 


3,830 


1,572 


1,601 


1,424 


1,246 


812 


784 


641 


665 


47.1% 


46.9% 


45.6% 


39.1% 


81 .2% 


80.1% 


79.6% 


83.3% 


51 .7% 


49.0% 


45.0% 


53.4% 












Direct Placement Success: 


64.5% 


61 .4% 


64.2% 


64.2% 



The MAT 0002 pass rate improved 
slightly. 

Progression to MAT 0024 increased. 



Success in MAT 0024 increased, but 
lags direct placement students. 



The MAT 0020 pass rate decreased. 

Progression to MAT 1033 increased 
and is high. 

Success in MAT 1033 increased, but 
lags direct placement students. 



detail according to the needs and authority of particular 
users. Faculty have access to a variety of data relevant 
to their own courses. They can, for example, find grades 
and other data for their courses in any previous term. Nine 
terms of data are shown for most queries. Thus, trends in 
pass rates, withdrawal rates, success rates, and student 
ratings are displayed prominently so faculty can quickly 
see the effect that innovative strategies or curricular 
changes may have on student success in their courses. 

Chairs and academic administrators can use this 
system to view pass rates and other data for courses 
and individual faculty members. Having access to 
such an easy-to-use, powerful system allows for useful 
comparisons. If, for example, a faculty member is doing 
something innovative in teaching a course, the success 
rate in her course can be compared with that of the 
same course taught by other faculty. The information 
management system is geared primarily to the college 
leadership, and it is heavily used. While it does not track 
individual student data, it can yield results based on 
class- or faculty-level data. It also gives information on 
enrollments (including the mapping of enrollments by ZIP 
code), student feedback results, faculty productivity, and 
program review data. 

Focused research. Another way we share data is through 
focused research, in which a group of faculty, usually 
from a particular discipline, work with IR researchers 
in formulating research questions about student 
achievement. Our experience with focused research 
first began, again, with the math discipline. Typically in 



focused research, IR researchers approach faculty with 
a suggested list of questions that we believe would help 
them systematically explore a topic that affects their 
students’ success. The list is refined with input from 
discipline faculty to best identify student challenges and 
successes. Several questions included on the original 
focused research agenda for mathematics were: “How 
well are students performing in each course based on 
placement score ranges?”; “Which courses are students 
having the most difficulty passing and what are the 
success and withdrawal rates?”; “How are students who 
are repeating the course performing compared to students 
taking it for the first time?”; “How many students pass a 
college-level math course within two years and what is the 
progression path?”; and, “What special projects are being 
implemented to enhance success and are they working?”. 

Over the past two years, it has become increasingly 
common for campus departments or college-wide 
disciplines to approach the IR office (rather than vice versa) 
with questions that may spawn focused research. If more 
than one campus department in the same discipline begins 
to ask similar kinds of questions, we sometimes encourage 
all campuses to collaborate on a proposed research 
agenda. For example, we recently received questions from 
several campus science departments about how students 
who took introductory science courses perform later in 
advanced science or nursing courses. Instead of pursuing 
each question that was posed piecemeal, faculty from the 
entire science discipline convened with IR researchers to 
formulate a cohesive set of research questions about how 
introductory science courses impact the progression and 
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success of science students. Over the past several years, 
we have found that focused research is an effective means 
to involve faculty ideas in college-wide research. 

Internal grants. In 2001 , MDC initiated an internal grant 
process to encourage discussion of student success 
and the identification of specific strategies to address 
issues of concern. Faculty apply for Learning Innovations 
Grants; student services staff apply for Student Services 
Innovations Grants. Applicants submit a proposal on 
a problem they want to address, what strategy and 
actions they plan to take, what outcomes they hope to 
achieve, how they will evaluate results, and how much 
money they will need. A panel reviews applications and 
chooses the most promising ones. Participation in these 
grants has increased dramatically. In 2006, 20 Student 
Services proposals were selected, including one on 
providing supplemental instruction in mathematics for 
targeted groups and one on mentoring. The 15 Learning 
Innovations Grants recently funded include one that offers 
a “Mathematics Problem-of-the-Week” contest and several 
involving learning communities. 

Step 5: Identify strategies 

Primarily through the roundtables and retreats 
described in Step 3, several strategies were identified to 
improve student learning and student progression along 
the developmental/early college-level math pathway. 
Faculty began to implement some practices based 
on these strategies before the college began formal 
discussions in 2003 to develop a QEP for the college’s 
SACS accreditation review that occurred a year later. 
Indeed some of the strategies piloted early on led to 
more formalized QEP strategies. (It is also the case that 
enthusiastic faculty continue to pursue strategies in 
addition to those included in our formal QEP.) While our 
initial investigation of developmental math took place 
before we devised our QEP, the processes set in motion 
by the QEP, and the fact that these processes and their 
results would impact the status of future accreditation 
reviews, served to strengthen the resolve of MDC 
faculty, staff, and leadership to seek out long-term 
strategies for improving student success in high-risk 
mathematics courses. 

The SACS Quality Enhancement Plan. MDC and all 
other SACS-accredited colleges are now required to 



develop and use a QEP. SACS established the QEP 
framework as a requirement for re-accreditation in 
order to facilitate continuous quality improvement at 
all its participating institutions. A college creates a 
QEP based on a faculty-led, comprehensive analysis 
of the effectiveness of each college’s student learning 
environment and the overarching college mission. Through 
this analysis, the faculty identify a topic associated with 
student learning that they want the college to focus on. 
Based on this topic, the QEP is used to chart a course 
of action for institutional improvement based on data- 
driven decision making steps. In subsequent years, each 
college is expected to show evidence establishing that 
improvement has occurred. 

MDC’s Quality Enhancement Plan. Consistent with our 
ongoing investigations, MDC faculty chose to address low 
student success in high-risk developmental and college- 
level math courses as the topic of MDC’s QEP, which 
is titled Student Success at Miami Dade Coiiege: The 
Mathematics Connection (Miami Dade College, 2004). The 
specific goal of our QEP is “to enhance student learning 
by developing innovative curricular, instructional, support, 
and assessment strategies in the high risk courses 
of College-Prep Mathematics, College-Prep Algebra, 
Intermediate Algebra, and College Algebra.” (p. 1). The 
period of implementation for the QEP is from 2004 through 
2008. A 38-member QEP team, which included 1 0 math 
faculty, 10 faculty from other disciplines, student services 
personnel, chairs and deans, and students, was then 
selected. 

Qnce we entered the QEP process, faculty committees 
were formed to write analytical reviews of relevant 
research and to visit other colleges engaged in strategies 
of interest. The faculty then presented several strategies 
that they wanted to try — linked courses (similar to the 
“math across the curriculum” approach), improvement 
of math advisement, and math lab tutor certified training, 
among others. 

Frequent testing strategy. Qne important strategy they 
wanted to try was establishing frequent testing in high-risk 
math courses. The selection of this strategy was based on 
research for the QEP as well as by internal college data. 
Students in the focus groups had previously highlighted 
that attendance is critically important in math courses and 
that falling behind can occur quickly. Some students felt 
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Table 4 

Pass Rates, Success and Withdrawal Ratios 

for the Frequent Testing Pilot Conduoted In Spring Term 2004 


Group 


N 


Pass Rate 


Success Ratio 


Withdrawal Ratio 


Control 


200 


0.545 = 54.5 % 


0.686 = 68.6 % 


0.205 = 20.5 % 


Experimental 


205 


0.648 = 64.8 % 


0.751 = 75.1 % 


0.137 = 13.7 % 



Notes: The success ratio is calculated by adding the total number ot A, B, and C grades and dividing by the total number 
of all grades minus the number of withdrawals, i.e., Suocess ratio = (A+B+C) -r (all grades - withdrawals). The withdrawal 
ratio is calculated by dividing the number of withdrawals by the number of all grades, i.e.. Withdrawal ratio = (withdrawals) 
(all grades). Since the pass rate is equal to the number of A, B, and C grades divided by the total number of all grades, 
including withdrawals, it follows that Pass rate = (1 - withdrawal ratio) x (success ratio). 



that having more frequent tests 
would be helpful to reinforce and 
confirm that they really understood 
prior instruction. 



Faculty accepted that idea and 
while conducting research for 
the QEP discovered that other 
institutions had found frequent 
testing to be useful in providing 
regular, consistent feedback 
to students and, ultimately, in 
improving outcomes. The math 
faculty also conducted a pilot program of frequent testing 
in a high risk math course on MDC’s Kendall campus. In 
that pilot program, about 200 students were randomly 
selected to attend sections of Intermediate Algebra in 
which 1 5 short tests — in place of the more traditional 4 
longer tests — were administered throughout the term. 
This “experimental group” had better pass rates, success 
rates, and withdrawal ratios than did those students 
in the “control group,” which consisted of about 200 
students who attended sections of the same course in 
which the 4 longer tests were administered (see Table 4). 



students through their email accounts on file with the 
college (assessment by comparing experimental and 
control groups). 

8. Establish a program to incorporate math into other 
disciplines (assessment by comparing experimental 
and control groups). 

9. Provide math faculty training on accommodating 
different learning styles (assessment by comparing 
pre- and post-training ratings). 

10. Upgrade the math support labs (assessment by 
comparing pre- and post-training ratings). 



Ten QEP strategies. MDC’s QEP identified a total of ten 

strategies, along with assessments for each (Miami Dade 

College, 2004, pp. 59-60). 

1 . Establish a program of frequent assessment 
(assessment by comparing with historical data). 

2. Provide learning prescriptions for students who 
demonstrate need (assessment by comparing 
experimental and control groups). 

3. Establish mathematics advisement procedures 
(assessment by comparing with historical data). 

4. Provide supplemental instruction for all students 
repeating college prep mathematics or college prep 
algebra (assessment by comparing experimental and 
control groups). 

5. Implement a CRLA-certified tutor training program 
(assessment by comparing pre- and post-training 
ratings). 

6. Provide math success skills training for all students 
repeating college prep mathematics or college prep 
algebra (assessment by comparing experimental and 
control groups). 

7. Send personalized interim progress reports to 



Interest of other disciplines. It is interesting to note that 
in the past few years, the systemic and data-based 
approach to improvement that was undertaken in math 
has been spreading to other disciplines and programs. 
The math faculty, who in years past had to endure 
significant negative attention, are now seen as innovative 
and progressive by other faculty who are calling on their 
own disciplines to engage in the same kind of systematic 
methods for improving curriculum, pedagogy, and 
supports for students in their courses and programs. 

Step 6: Implement and assess strategies 

We insist that everyone who wants to implement a 
new improvement strategy at MDC have some kind 
of assessment plan to test how well that strategy is 
working. To facilitate appropriate assessment, we strive 
to make the IR office very accessible to the faculty. 

We often get queries from faculty who want our Input 
as they pilot new programs. The IR office helps create 
assessment tools for interventions of all kinds. We 
consult, for example, with all the Innovations Grant 
awardees to make sure that they have good evaluation 
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plans in place and to ensure that data they may require 
from the IR office for their assessments are ready when 
they need them. 

Leading QEP implementation. Faculty committees 
headed by one lead faculty member are in charge of 
implementing each strategy described in our QER 
Having faculty, as opposed to administrators, in charge 
of implementation is helpful in gaining cooperation from 
others and in maintaining the QEP as a faculty-driven 
plan. We have found, however, that it is also important 
for the lead person to possess the authority to make 
things happen. Some faculty are unaccustomed to or 
uncomfortable in performing the tasks that are required 
by this role, such as asking for additional funds when 
needed or directing other faculty. This has contributed to 
inconsistent implementation of some strategies. 

We are currently trying to find ways to help faculty 
committee leads fulfill their demanding roles. And we 
have recently hired a faculty member to be in charge 
of implementation of all QEP strategies. Released from 
her teaching responsibilities, she will be working with 
the IE office, the QEP oversight committee, and the 
math faculty to help ensure that the QEP strategies are 
implemented and evaluated as they were intended. 

Preliminary results. For several of the QEP strategies 
undertaken, we have preliminary results about their 
impacts. For example, the implementation of tutor 
training has shown improvements in lab usage and in 
the helpfulness of the math lab tutors. Student surveys 
conducted in 2006 before and after training indicate that 
the lab usage rate increased by 7 percentage points 
(from 84% to 91 %). And the frequency of daily or near- 
daily lab use by students increased by 5 percentage 
points (from 36% to 41 %). While student satisfaction 
ratings of tutors were generally high before the training, 
ratings on two specific items increased by 3 percentage 
points: 1) the tutor was knowledgeable about the 
technology available in the lab (increased from 95% to 
98%) ; and 2) the tutors were able to provide related 
information and/or suggestions about study techniques, 
note-taking skills, etc. (increased from 89% to 92%). 

Regarding improved math advisement, a higher 
proportion of students are taking their first required 
math course earlier and in their first term than before the 



strategy was implemented. College-wide, 79 percent 
of students took mathematics during their first term 
in 2006 compared with 70 percent in 2003. Likewise, 
more students are taking the next math course in their 
prescribed sequence immediately after completing 
the prerequisite course. For example, 77.5 percent of 
students took MAT 0024 immediately after completing 
MAT 0002 in spring 2006 compared with 74.1 percent 
in 2003, and 83.3 percent took MAT 1033 immediately 
after completing the prerequisite in 2006 compared with 
81 .2 percent in 2003. (These results are shown on page 
15 in Figure 1 , which is an excerpt from an MDC briefing 
package.) As described below, we have had mixed 
results with the frequent testing strategy. 

Implementation integrity. We have come to find that 
implementation integrity is very important. Qur experience 
with the frequent testing strategy made that very apparent. 
When we first began to implement frequent testing in four 
high-risk math courses throughout the college in 2004, 
most faculty were enthusiastic. Many faculty agreed to 
incorporate the strategy, and instructors for the courses 
claimed to be using it once the term was underway. Yet 
during our first round in evaluating the associated course 
outcomes of students, we encountered some strange 
results. The strategy seemed to be working for some 
sections of the courses but not for others. For example, 
pass rates were higher for half of the frequent testing 
sections of MAT 0020 in 2005 (two were significantly 
higher). For MAC 1105, pass rates were significantly 
higher in six of the frequent testing sections, while five 
were significantly lower. The results we observed were so 
odd that we began to ask instructors for more specific 
information about how they were, in fact, implementing 
frequent testing. 

As it turned out, the faculty were not following a consistent 
method In implementing the strategy. Some instructors 
were incorporating ten additional tests in their sections 
while others were adding only four or fewer additional 
tests. Equally important, while some Instructors discussed 
the results of the tests with their students in great detail, 
other instructors engaged in little review of the results. 
Thus the frequent testing strategy was not implemented 
with great integrity in the first round. 

We have experienced a similar problem in the 
implementation of other strategies. Learning 
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communities, for example, is an intervention that has 
been adopted by several disciplines. Yet some faculty 
who claimed to be involved In learning communities 
have not really Integrated the curriculum of the linked 
courses. Nor have their participating students met to 
discuss themes common to the two courses, which 
would be expected In a learning community environment, 
it seems that we have a wide variety of linked courses 
and learning communities. Understanding the differences 
between them is important to assessment of the strategy. 

Continuous improvement. We refer to the final aspect 
of implementation assessment as “closing the loop.” 

It is a key feature of the overarching student outcomes 
improvement process. When results are obtained 
about how well a strategy is working, those results 
should then be used to shape how that strategy 
can be improved. Once we had some results on the 
frequent testing implementation, for example, we were 
compelled to act on that information. We used the 
results of the first round evaluations to help decide how 
to change the implementation to make it more effective. 

It is easy to see how “closing the loop” in a series of 
implementation cycles corresponds to the notion of 
continuous improvement advocated by SACS and other 
accreditation agencies. 

Step 7: Establish accountability 

It is easy to get excited by new interventions and reforms 
aimed at improving the quality of education for students. 
After the implementation of new practices has begun, 
however, it is also easy for faculty and staff to lose focus 
as they get overwhelmed by other responsibilities and 
projects. We wanted to be certain that the initial vigor of 
MDC’s improvement efforts would be maintained over 
the long run. While the college’s QEP and accreditation 
reports go a long way In assuring that this happens. In 
2003 the college also adopted Internal accountability 
requirements for all MDC disciplines, schools (collections 
of allied programs, such as the School of Allied Health), 
and student service areas. 

Annual reports. Each of these entities must prepare an 
annual report In which they articulate the purpose of 
their discipline, school, or area and what their expected 
outcomes are for students enrolled in their courses and 
programs. They must also tell how they measure such 



outcomes. Finally they report on the strategies that they 
are using to improve outcomes and how well, based on 
data, those strategies are working. 

Each annual report Is then given to a campus dean 
for review. At MDC, one designated campus is 
responsible for the college-wide coordination of a 
particular discipline. For example, the Kendall campus 
is responsible for the math discipline. So the dean of 
the Kendall campus reviews and may make suggestions 
to the math discipline’s annual report before that 
report goes to the campus president for final approval. 
Each report then goes from the campus president’s 
office to the IE office, where it becomes part of our 
documentation on college Improvement. The college’s 
institutional effectiveness committee reviews the reports 
and provides feedback to each discipline. 

Strategic plan. The annual reports are an integral part 
of MDC’s strategic plan, which includes additional 
accountability measures specific to the college’s 
long-term goals. One of five themes in the MDC 
strategic plan focuses on barriers to student success 
and following data-driven processes to identify and 
address particular challenges. The same theme Includes 
objectives related to the assessment of Institutional 
interdisciplinary learning outcomes. Strategic plan 
measures and associated data speak to how well the 
college is meeting its goals. 

Step 8: Institutionalize the process 

In this paper, we have already described methods 
that are used to share knowledge of what strategies 
are underway and how well they are working. The 
strongest of these methods In terms of campus-wide 
dissemination among decision makers with the most 
authority is probably the annual reporting process. As we 
have stated, all the Information in the annual reports is 
reviewed by each coordinating campus dean. Highlights 
are then shared with the MDC executive committee so 
that everyone is aware of the key strategies that faculty 
are pursuing, what impacts the strategies have had, and 
how they match with the college’s strategic plan. 

Venues that support institutionalization. Sharing such 
information also encourages others to get Involved and, 
ultimately, to Institutionalize the process of systematic 
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change. In addition to regularly scheduled meetings 
and retreats already discussed, faculty also participate 
in a professional development day once a year. In the 
morning of this day, concurrent sessions are held in 
which faculty members may present improvement 
strategies they are undertaking. During these sessions 
faculty often showcase the successful strategies they 
have implemented and the results they have achieved. 

In the afternoon of the professional development day, 
discipline discussions take place. These have proven to 
be good venues for sharing and learning for faculty from 
the same discipline who reside on different campuses. 
For the future, the college will be developing a “faculty 
institute” in which faculty can get more involved in the 
study and evaluation of teaching and learning practices 
within their discipline, including those practices impacted 
by the specific strategies they are pursuing. 

Faculty and staff evaluations. All these venues 
provide for informal and formal conversation among 
administrators and faculty that is instrumental for the 
institutionalization of student outcome improvement 
strategies as discussed in this paper. Finally, faculty and 
staff evaluations also play a role in soliciting support 
for this kind of college improvement. During evaluation 
reviews at MDC, faculty, staff, and administrators are 
asked to identify personal goals and to reflect on how 
their work advances the goals of their discipline, the 
QEP, the strategic plan and the mission of the college. 

By including this line of discussion in faculty and staff 
evaluation reviews, the college emphasizes the priority 
placed on strategic goals and on the improvement of 
student outcomes. 

Conclusion: Lessons Learned So Far 

As we have strived to make clear throughout this 
report, the use of data is crucial not only for developing 
appropriate strategies to address problems such as 
poor student progression, but also for determining if 
such strategies are effective, and for modifying them 
when necessary. We have found that sharing data 
with stakeholders throughout the college also helps to 
establish a “culture of inquiry and evidence” in which 
such data become a focus for discussion and action 
about student learning and achievement. Many of the 
lessons we have learned over the past several years are 
associated with how student data is utilized throughout 



the college. These lessons represent for us the key points 
to keep in mind for supporting institutional change that 
increases student success. 

Start with available data and dig deeper as needed. 
Although the root causes may not be adequately 
understood, at many community colleges, key problems 
that thwart student success are probably well-known. At 
MDC, we knew that the developmental math pathway 
was not working well. By gathering more data related 
to that problem, discussions and useful disagreements 
arose that helped us better diagnose causes and 
choose interventions. The availability of additional data 
also helped to generate more interest in improving the 
situation. 

Use a variety of measures and methods. Different kinds 
of data can be useful in gaining insights. Cross-sectional 
analyses are useful and generally easy to conduct. 
Longitudinal cohort analyses can indicate where 
particular groups of students encounter difficulties. 
Benchmark comparisons can help a community college 
understand its strengths and weaknesses vis-a-vis like 
institutions. At MDC, we found that allowing student 
voices to be heard through student focus group data 
was very important in understanding challenges in the 
developmental math pathway. 

Share, listen, and collaborate with stakeholders. 
Communicating openly is very important for institutional 
researchers. At MDC, we encourage people to call and 
tell us if they think we are analyzing data poorly or if we 
are producing errors in our reports. By doing so, we can 
learn if we have made mistakes. And if through dialogue 
and investigation we learn that we have not done so, 
then the issue of possible error can be eliminated, and 
users of institutional data can better trust our results. 
MDC institutional researchers also participate in faculty 
meetings throughout the college. By participating in 
meetings of faculty and staff, researchers act more as 
“insiders” and helpers rather than as “auditors.” 

Establish assessment plans before implementing 
strategies. In order to understand if and how well a 
strategy is working, a method of assessment must be 
established before implementation begins. Appropriate 
assessment plans are necessary to determine if a 
strategy is working and whether or not modifications 
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should be made after it has been implemented for one 
or more cycles. At MDC, the assessment of the college’s 
frequent testing strategy alerted us to the fact that the 
strategy was not being implemented as designed by all 
instructors. The implementation was therefore modified 
to promote greater implementation integrity. 

Channel the enthusiasm for new ideas. Faculty are 
very creative, and once momentum for improvement 
builds, faculty may continue to develop new ideas to 
help students after core strategies have already been 
established. This is, of course, not to be discouraged. But 
it is important that the enthusiasm for new interventions 
does not interfere with the implementation or evaluation 
of strategies that are already underway. At MDC, we 
reminded faculty who wanted to implement additional 
strategies after our QEP was finalized that we may first 
want to see how well those QEP strategies are working. 

Integrate innovation with existing college processes. To 
the extent that it is possible, working within the existing 
institutional structure is often easier than establishing 
new or separate processes. At MDC, we tried to 
incorporate our student outcome improvement strategies 
into the regular functioning of the college. We used the 
standard review processes associated with the strategic 
plan, the core indicators report, and annual reporting as 
channels for proposed innovations that required different 
kinds of approvals. For the most part, we worked within 
the established college-wide structure. 
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